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\ (57) Abstract 

A real-time surgical simula- 
tion system includes a physical body 
environment, a body environment 
simulator and a surgical tools sim- 
ulator. The physical body environ- 
ment replicates a portion of a hu- 
man body and includes a synthetic 
skin layer. The synthetic skin layer 
allows insertion of a surgical tool 
through the synthetic skin at an ar- 
bitrary location. The body environ- 
ment simulator simulates a body en- 
vironment within the physical body 
environment. The surgical tools 
simulator simulates at least one sur- 
gical tool within the virtual body en- 
vironment mat can interact with die 
virtual body environment, and that 
is coupled to a surgical tool that is 
manipulated by a user of the surgical 
simulation system. The simulated 
surgical tool moves corresponding 
to movement of the surgical tool by 
the user. The surgical simulation 
system further includes an interac- 
tion simulator that simulates inter- 




system further includes an interne- . , atMl -y^ai tool. This simulated 
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of surgical procedures such as arthroscopy. 
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management decisions. More recently, medical education programs have used computer 
graphics for instruction of a patient's anatomy. These programs provide the advantage of 
giving a student an opportunity to access anatomic images and related textural information, 
rather than spending numerous hours dissecting cadavers in a lab, and provide the additional 
benefit of allowing the student to spend as much time as needed to learn me material. 

Although these computer-assisted programs offer many advantages in medical 
education, there are many limitations and drawbacks such as, for example, there is a limited 
scope of depicted anatomical information, and these programs do not allow for practice of 
actual medical procedures under realistic conditions. 

One proposal that has been made to simulate living conditions in non-living 
substitutes is U.S. Patent No. 4,907,973, granted on March 13, 1990 to David C. Hon 
(hereinaiterthe Hon patent). Referring to Fig. 1, the Hon patent discloses a computer-based 
surgical simulation system 100 including a peripheral device designed to simulate a surgrcal 
instrument, as well as the involved anatomy of a patient In particular, the Hon patent 
discloses in one embodiment a system for computer simulation of an endoscope examination. 
The Hon system includes a mock catheter 1 02 that is inserted within a model 1 04 or mock 
arterial path of a physical internal body part of a model. Within the model, there are a 
plurality of sensors (not illustrated) on the mock arterial path that track the progress of the 
catheter through the mock arterial path. The sensors transmit corresponding signals 105 to a 
computer 106 which responds to the signals by accessing stored video information in storage 
medium 109 representing the view which would be observed from the relative location of a hp 
of the catheter device. The video representation 107 is then presented by a video display 109 
to the user 1 10 of the system of the Hon patent. In one embodiment of the system of the Hon 
patent, a vessel constricting simulator within the mock arterial path provides a fixed resistance 

to the progress of the catheter. 

While the system of the Hon patent has the potential for providing a feel of a catheter 
device, it suffers a number of infirmities. For example, the system of Honuses a portal for the 
surgical instrument that is fixed to the physical mannequin. The system of the Hon patent 
also lacks the flexibility to reatistically simulate changes in the mock environment response 
to manipulations and performance of the surgical instrument by the operator of the system. 
By way of example, the system of the Hon patent does not provide for the realistic simulate 
of bleeding, e.g., in the event an arterial wall is punctured by the catheter, or catheter balloon 
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Mation and consequent removal of an occlusion within the mock artenal path. Inaddition, 
the displayed indicia representing the view from the catheter is a prestored video sequence. 

Moreover, the system of the Hon patent does not provide for tracking and displaying 
rotation of the catheter device in the mock body environment; can only apply passive braking 
forces along the axis of the catheter by the constrictors along the mock arterial path and 
therefore cannot simulate any to^^nt^toP^*-*^**"*" 
esophagus ofme body such^ * 
addition, the image representing the internal body environment cannot be m 
toeasaresmtofanymteractionofmecameterwith^ 

therefore simulation of interaction such as, for example, deformation of tissue through 
interaction with the catheter is not possible. The system of Hon does not have the abihty to 
simulate interaction of the mock body environment with the surgical tools by changing the 
mock body environment shape, appearance, or position in response to any of poking, 
plucking, puncturing or incising of the mock body environment 

The Hon patent also discloses that surgical performances of the user can be recorded 
for evaluation, review, and comparison to other performances. However, because the system 
of the Hon patent suffers the infirmities discussed above, the system of the Hon patent cannot 
record, compare, evaluate and score performances of a user based on forces of interaction 
between the body environment and the surgical tools. 

Another proposal for a system for providing a realistic simulation of a cardiac 
catheterization procedure is disclosed in international publication No. WO 96/28800, 
published on September 19, 1996 (the Merril disclosure). Referring to Fig. 2, the medical 
simulation system of the Merril disclosure includes a computer-human interface device 1 1 1 
and a computer 1 1 3 storing a software program which works in conjunction with the 
computer-human interface device. In a preferred embodiment of the Merril disclosure, the 
system simulates a cardiac catheterization procedure, and the interface device is a catheter 
interface device that tracks the translation* and rotational movement of a mock catheter 121, 
and provides a signal 123 to the computer program. The computer program generates a tactile 
feedback force signal 125 that is output to the catheter interface device to simulate resistance 
to the mock catheter. The computer program also provides a virtual reality simulation of 
movement of the catheter within the virtual catheter on a high-resolution monitor 127. The 
computer program can provide a variety of enhancements including 3-D imaging, narrative 
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audio or music via a CD-ROM 124, simulated radiopaque dye infusion, bleeding and atrial 
pulsation in synchronism with an on-screen EKG display. 

While the Merril disclosure provides a more realistic simulation than that of the Hon 
system, it also suffers a number of infirmities. In particular, the system of Merril uses a portal 
for the surgical instrument that is fixed. Furthermore, the system of the Merril disclosure does 
not provide for simulated objects and tools within a virtual body environment that have their 
own position, orientation, space, and that can move independently in the virtual body 
environment. The system of the Merril disclosure does not provide for contact between any of 
the simulated objects and tools within the virtual body environment Moreover, the system of 
the Merril disclosure does not allow for the catheter to have a position independent of the 
virtual body environment and that can be moved independent of the body environment. 

The system of the Merril disclosure is also limited to "canned" or pre-stored 
simulations of simulated vessel movement, simulated catheter positioning and simulated 
vessel expansion in response to simulated angioplasty balloon inflation. These are 
precomputed responses that are not responsive to any interaction between the catheter and the 
simulated arterial tree. In addition, the system of the Merril disclosure can only provide 
restrictive forces to the mock catheter, and therefore suffers the same infirmities with respect 
to simulating realistic forces of the simulated surgical procedure as discussed above with 
respect to the Hon patent 

The surgical tools 220 employed in minimally invasive procedures such as arthroscopy 
and laparoscopy typically have at least four degrees of freedom (See Fig. 3). The surgical tool 
can move in and out of a portal 222 along a longitudinal axis 226 of the tool (axial 
displacement), rotate about the longitudinal axis of the tool (axial rotation) 228, and rotate 
about two axes 230, 232 in the plane of the body (transverse rotation). Additional degrees of 
freedom may include those needed for jaws or graspers of the tool. A system for simulating 
these interactions between a surgical tool and the body requires force feedback along one or 
more of these degrees of freedom. A force feedback tool such as that of the Immersion Corp., 
2158 Paragon Drive, San Jose, California (Website http//:www.immerse.com) and its 
Laparoscopic Impulse Engine product has simulated one or more of these force interactions 
with motors or other force or torque producing devices that apply axial displacement forces 
and transverse rotation torques. In particular, the Laparoscopic Impulse Engine includes 
motors or force or torque producing devices, that are co-located with the portal, and requires 
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anchoring these devices to a physical body mock up. However, a drawback of the 
Laparoscopic Impulse Engine is that a motor is required on the force feedback device that is 
coincident with the center of rotation of the tool and that is coincident with the portal, that is 
used to simulate the resultant moment. Another motor that drives an axial displacement of the 
tool is also needed to simulate the axial force. In addition, the motor arrangements need to be 
anchored into the body part at a point of the portal to allow the mechanical body part to 
support the skin component forces of the tool. This system is therefore complex and 
expensive. 

A Knee Arthroscopy Training System of McCarthy and Hollands (ref. McCarthy, 
A.D., and Hollands, R.J., "A Commercially Viable Virtual Reality Knee Arthroscopy 
Tuning System", Medicine Meets Virtual Reality, Westwood, Hoffrnan, Stredney, and 
Weghorst (Eds.), IOS Press and Ohmsha, 1998) does not use force feedback at all. Their 
system employs an articulated physical leg and an arthioscope. The position and orientation 
of the leg and scope are tracked so that corresponding computer graphic images of the view 
from the arthroscope can be computed. However, the lack of force feedback between the 
arthroscope and probe and the bones and tissues inside the leg make the system difficult to use 
and less than realistic. 

Sammag Of *H Invention 

In view of the foregoing, it is an object of the invention to provide an improved system 
for simulating minimally invasive surgical procedures and for training. 

According to one embodiment of the apparatus of the invention, a surgical simulation 
system includes a physical body environment that replicates a portion of a human body and 
includes a synthetic skin layer. The physical body environment is configured to allow 
insertion of a inimmally invasive surgical tool, such as those used in arthroscopy or 
laparoscope through the synthetic skin layer into the physical body environment at an 
arbitrary location (the portal) selected by a user of the surgical simulation system. The 
surgical simulation system also includes a body environment simulator that simulates a virtual 
body environment within the physical body environment The surgical simulation system also 
includes a surgical tool simulator that simulates a simulated surgical tool within the virtual 
body environment and that interacts with the virtual body environment. The simulated 
surgical tool is simulated at a location inside the virtual body environment corresponding to 
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I. arbitrary location that te physical tool is inserted into the physical body environment and 
is simulated so that the simulated surgical tool moves eonesponding to me surgical Ml *at . 
manipulated by the user of the stugical simulation system. The surgical simulation system 
also includes an interaction simulator that simulates interaction between the virtual bod, 
environment and the simulated smgical tool, including any foree or torque excused between 
the simulated surgical tool and the virtual body environment The interaction sunulator 
provides a mn^imensional (3-D) force vector at an output that represents the simulated 
interaction. The smgical simulation system also ir*ludes a force feedback device mat appl.es 
me 3-D force vecterr a, a point on the surgical too. outside of the physic*, body environment 
and separate from the portal. The surgical simttiation system also includes a visual disphty mat 
displays the virtual body environment, the simulated smgical tool, and the simulated 
interaction between the virtual body environment and me simulated surgkal tool. Wrth tins 
arrangement a realistic simuhued smgical procure is provided to the user to learn, practice, 
and experience various minimally invasive smgical procedures. 

A preferred embodiment of the surgical simulation system also includes a resrbent 
sheet materia, disposer, beneath foe synmetic skin layer. With mis arrangement the portal 
piays the role of a amble pivot roreiug the mo. to rota* about the pivot when appropriate 
.mnsve^romtionforcosamprt-videdbymeforeefeedbackdevice. This arrangement 
obviates (be need for a motor ortorque source to be located at the pivot pomt 

The preferred embodiment of tire surgical simulation system further includes a 
petspectivedeterrmmng device thmdetennines a position of me smtulateti surgical tool wttir 

for displaying the virtual body environment and the simulated surgical mo. from a perspecOve 
of the physical tool within the physical body environment 

According to one embodiment of a method of simulating minimally invasive surgery 

according to tire invention, the mefltod comprises the steps providing a physical body 

envirormrentti^rep.icatesaportionofah^ 

vrfmin the physica. body environment simulating a surgical mo. within the virtual body 
envhonmenti simulating interaction between (he virtual body environment and the annotated 

simmamd interaction. Ttie physical body environment is provider, tit tdlow insertton of a 
surgical tool through a synthetic skin layer into the physical body environment at an arbitrary 
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■ i^bvauser In addition, «he simulated surgical tool * *—* * • ta " ta 
locauon selected by a user, m au b 

a, a point oms.de of foe ^ various urinal, 

surgical procedure is provided to the user to i«r , v 

iavasive surgical procedures. environment 
A preferred embodiment of the step of providing the physical body ™° 

^^^^^^rbeuseof^^asa-e 
•h™. of me simulated surgicd tool with respect to the virtual body 

^rz." Zi-S— 

grielDsiciilllio^^ 

^ f of the present invention will become apparent from the 

foUowing detailed description when taK ^Hon onlv and are not intended as 

a definition of the limits of the ^j™^"^^^^^^ following detailed description of 

art; v w^nart of a surgical simulation system according to the 

Fig. 2 illustrates another embodiment ot a surgi«u 

prior art; 
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Fig. 3 illustrates typical degrees of freedom involved in minimally invasive surgical 
procedures; 

Fig. 4 illustrates a preferred embodiment of a surgical simulation system of the 
invention; 

Fig. 5 illustrates a perspective view of the mechanical leg of the surgical simulation 
system of Fig. 4; 

Fig. 6 illustrates a frequently used position of the mechanical leg of Fig. 5; 

Fig. 7 illustrates an underlying sheet and top layer making up synthetic skin of the 

mechanical leg of Fig. 5; 

Fig. 8 illustrates a view from the arthroscopic tool of the virtual body environment; 

Fig. 9 is a flow diagram of a surgical skill evaluation method of the surgical simulation 
system of Fig. 4; 

Fig. 10 illustrates an embodiment of a surgical skill evaluation module of the surgical 
simulation system of the invention; 

Fig. 1 1 illustrates a method of evaluating a user's overall performance of a simulated 
task with the surgical simulation system of the invention; 

Fig. 12 illustrates a method of measuring any surface damage to the virtual body 
environment by a user of the surgical simulation system of the invention; 

Fig. 13 is a block diagram of an embodiment of a processor of the surgical simulation 

system of the invention; 

Fig. 14 is a block diagram of an embodiment of a 3-D graphic simulator within the 

surgical simulation system of the invention; 

Fig. 15 illustrates multiple forces that result from a surgical tool being placed through 
a portal in a skin of a body and that interact with the organs, tissues and the like of the body; 

Fig. 16 illustrates how the simulated interaction forces according to the invention, are 
decomposed into one component along the axis of the tool and another component that is 

transverse to the axis of the tool. 

Fig. 17A aiustrates a cross-sectional view of deformation of a simulated body object 
according to a deformation module of the surgical simulation system of the invention; 

Fig. 17B illustrates a perspective view of the deformed body object of Fig. 17A; 

Fig. 1 8 is a block diagram of an embodiment of a dynamic simulator within the 
surgical simulation system of the invention; and 
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Fig. 19 illustrates a method of movement modeling by the surgical simulation system 
of the invention. 

Qelailsd description 

The method and apparatus of the invention may be used, for example, as a surgical 
training simulator for niinimally invasive orthopaedic surgery and, as will be described below, 
includes a jointed mechanical body part and a computer-based simulator that in concert 
emulate the surgical tools and the virtual body environment of the nunimally invasive surgery. 
Although the preferred example to be described below will be that of arthroscopic surgery, it 
is to be appreciated that this method and apparatus can be used for any inimmally invasive 
surgery. It is to be understood that for this application, an active force-feedback device is a 
device that provides forces to surgical tools connected to the active force-feedback device. 

A preferred embodiment of a surgical simulation system 10 of the invention is 
illustrated in Fig. 4. The surgical simulation system includes several elements that work in 
concert including: a simulation processor 12 such as, for example, a personal computer, a 
graphics processor 13 and a display 14 such as, for example, an Octane computer graphics 
workstation and monitor made by Silicon Graphics, Inc.; a computer communications network 
15 connecting the two processors, such as, for example, an ethernet; a jointed mechanical 
body part 47, force-feedback devices 16, 18, manufactured by, for example, SensAble 
Technologies of Cambridge, Massachusetts that provide active force feedback; and surgical 
tools 20, 22 such as, for example, an arthroscope and arthroscopic probe. It is to be 
appreciated that the simulation processor and computer graphics processor can exist in either 
multiple distinct computers as described above, in one computer with multiple processors or 
even in a single computer processor. It is also to be appreciated that although an arthroscope 
and probe are illustrated as the surgical tools, any surgical tools are within the scope of the 
invention and can include, for example, those used in laparoscopic surgery. 

Referring to Fig. 5, in the preferred embodiment, the mechanical body environment 47 
is a leg comprised of a universal joint 191 simulating a hip joint, a rigid link 192 covered with 
flesh-like synthetic material to serve as a thigh, a knee joint 193, a shaped rigid shell that 
serves as a supporting underlying surface of the knee, and a rigid link covered with flesh-like 
synthetic material to serve as a lower leg and foot 195. Sensors (not illustrated) attached to 
each of three rotational axes of the hip joint and the single rotational axis of the knee joint 
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measure the orientation of the thigh and lower leg with respect to a fixed reference platform. 
The leg can be moved and manipulated by the hands of the user in the normal ways a human 
leg is manipulated during a typical arthroscopic examination. This manipulation is used as an 
integral part of the simulated examination to reveal places otherwise hidden from view by the 
simulated arthroscopic camera during a simulated examination. 

In particular, the mechanical leg 47 can be positioned into the position illustrated in 
Fig. 6, and that is frequently used to apply stresses to the knee in order to open up lateral 
compartments in the leg. The knee joint 1 93 is hollow in the place the arthroscopic tools 20, 
22 are to be placed. The rigid shell in the shape of the knee serves as an exoskeleton type 
structure that maintains the exterior appearance and feel of the knee. It is covered with a 
fleshy material that will be described in detail below. While this shell is rigid, it can be 
punctured in arbitrary locations with the typical surgical tools in order to create a portal for 
insertion of the surgical tools. The knee exoskeleton is also strong so that it can support 
transverse forces that will be applied to it by the force feedback device 18 via the surgical tool 
20 in order to create simulated forces of interaction between the surgical tool and the virtual 
body environment The sensors (not illustrated) attached to each rotational joint (hip and 
knee) are used to determine the position and orientation of simulated anatomical structures 
inside the knee during the simulated surgical procedure. Additional force sensors (not 
illustrated) attached to the leg sense the application of varus and valgus forces to the leg by 
the user during an examination. These forces are sensed and the information is used by the 
surgical simulation system to reposition the simulated anatomical structures inside the knee in 
a realistic manner as a result of the user applied forces. 

The mechanical body part 47 is provided to look, feel, and move like a human leg. 
Referring to Fig. 7, in a preferred embodiment, the mechanical leg is covered with a synthetic 
skin 212. It is to be appreciated that although the mechanical leg is used for this preferred 
embodiment of the invention, that a mock-up of any body part can be provided and is intended 
to be within the scope of this invention. The interior of the mechanical leg is hollow in areas 
a physician would normally place the arthroscopic tools. In this way, the tools when inserted 
through a portal 210 in the leg are free to move in an unobstructed way inside the mechanical 
leg. The mechanical leg can be manipulated by the user's hands in the normal way an 
orthopedic surgeon would handle a human patient's leg during an orthopedic surgical 
procedure such as, for example, flexing or extending the knee or hip, abducting or adducting 
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the hip, external or internal rotating of the hip, and application of varus or valgus forces to 
reveal certain internal regions of the knee during an examination. 

The skin of the mechanical leg 47 includes a resilient sheet 214 made of a layer of 
reinforced nylon underneath a more compliant skin-like surface 212 made of a compliant 
layer of silicon. With the mechanical leg 47 of the present invention, it is possible for the user 
to make an arbitrary portal 210 in the skin 212 and sheet 214 when inserting the surgical tool 
20 into the mechanical leg. The underlying sheet is draw tight so it can provide lateral 
support to the surgical tool when the surgical tool is positioned through the sheet, thereby 
allowing the surgical tool to rotate easily about the point of intersection 216 (the pivot) with 
the sheet, when forces 217, 218 transverse to the longitudinal axis 215 of the surgical tool are 
applied at some distance from the pivot. The sheet provides the supportive pivot to the 
surgical tool, which is important for emulating forces of interaction between simulated objects 
inside the mechanical leg and the surgical tools. With the resilient sheet underneath the more 
compliant skin-like surface, the surgical tool can move in and out of the portal along the 
longitudinal axis 215 of the surgical tool (axial displacement) in response to an axial 
displacement force 219, rotate about the longitudinal axis of the surgical tool (axial rotation 
221), and rotate 227, 229 about corresponding axes 223, 225 in the plane of the synthetic skm 
at the pivot point of intersection with the surgical tool. However, the tool cannot easily 
translate at the point of intersection with the arbitrarily placed portal due to the resistance of 
the sheet This is also important for simulating the interaction between the simulated surgical 

tool and the mechanical leg. 

Referring to Fig. 8, there is illustrated a view that the user would see from the 
arthroscope tool 20 of the simulation system of the invention. In particular, Fig. 8 illustrates 
one view of a femoral condyle 60 and a tibia 62 from the perspective of the arthroscope. The 
user of the simulation system views the inside of the knee through the arthroscope after 
inserting the arthroscope at an arbitrary portal 210 into the mechanical leg 47 (see Figs. 6-7), 
as discussed above. Any interaction forces between the arthroscope (not illustrated) and the 
virtual body part are felt by the user with the surgical tools and by a 3-D force feedback vector 
(to be discussed infra) applied through the force feedback device. As will be discussed in 
detail below, these forces of interaction between the simulated surgical tool and the virtual 
body environment are calculated by a simulation processor (see Fig. 4) and conveyed to the 
user through the surgical tool by the force feedback device 16. In addition, the visual image 
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of the virtual body environment is determined by the graphics processor 13 and displayed to 
the user via the 3-D graphics display 14. Accordingly, the surgical simulation system allows 
the user to practice, for example, the surgical procedure of arthroscopy. It is to be appreciated 
that although Fig. 8 illustrates a virtual knee environment, the invention is not to be limited to 
arthroscopy and can include any surgical procedure such as, for example, laparoscope and 
that all such surgical procedures are intended to be within the scope of the term simulated 
surgical procedure, as defined by this invention. 

As will be described in further detail below, it is an advantage of the present invention 
that it is possible for the user to arbitrarily choose and change the portal 210 (see Fig. 6) for 
insertion of the minimally invasive surgical tool 20 into the mechanical leg 47. This 
functionality makes it possible to easily experiment with the effect of different portal sites. 
This is important, for example, when the user is creating a portal site with an incision as 
opposed to using a natural orifice of the body. In this manner, the choice of a correct place for 
incision of the surgical tool is one of the skills that a user of the system of the invention can 
experiment with and learn. This is one advantage of the present invention over the related art 
discussed supra, which requires a fixed instrumented portal to provide the instrumentation and 
force feedback required for simulation. 

Referring again to Fig. 6, the active force feedback device 18 coupled to the respective 
surgical tools 20 applies forces having a magnitude and direction provided as a three- 
dimensional (3-D) force vector, provided by an interaction simulator (to be discussed infra), at 
a point of attachment 21 1 of the force feedback device to the surgical tool. This point of 
attachment is outside of the mechanical body part 47 thus enabling the tool to be extracted 
from the physical body environment completely, moved about freely, then inserted into 
another portal 210 of the user's choice all the while remaining attached to the force feedback 
device. This is an advantage over the related art discussed above, which requires tools and 
portals that are fixed relative to the physical body environment. Each force-feedback device 
16, 18 (see Fig. 4) in coordination with sensors (not illustrated) that interface with the surgical 
tools 20, 22, also measures a position, orientation and configuration of the surgical tools. As 
will be described in greater detail below, the processor 12 simulates the simulated surgical 
tools within the virtual body environment (see Fig. 8). The simulated surgical tools have a 
position, orientation and movement that corresponds to the surgical tools. In addition, the 
forces applied to the surgical tool by the force feedback devices are used to simulate forces of 
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respective force feedback devices, and provide that information to the processor 12. As will 
be discussed in greater detail below, the simulation processor uses the position, orientation, 
and tool configuration information from the active force feedback device and surgical tool 
sensors to determine the position and orientation of the simulated surgical tools in the virtual 
body environment. This information is combined with position and orientation information of 
the virtual body environment to detect contact between the simulated surgical tools and the 
virtual body environment, and to compute the 3-D force vector to be applied by the force 
feedback devices. This 3-D force vector information is applied by the force feedback devices 

16, 1 8 to the surgical tools. 

The force-feedback devices 16, 1 8 when combined with physics-based simulation by 
the dynamic simulator 78 (see Fig. 13) of the body environment, provide realistic constraints 
on manipulation of the surgical tools within the virtual body environment. Accordingly, when 
the user of the system touches any component of the virtual body environment with the 
simulated surgical tools, the user will feel it by the forces presented to the surgical tools 
through the active force feedback devices. Simulated forces of interaction are also applied to 
the elements of the virtual body environment, which respond to the forces in realistic ways as 
provided by the physics-based simul ation of the virtual body environment in response to 
movement by the user of the tools 20, 22. 

Moreover, as will be described in greater detail below, the dynamic simulator 78 (See 
Fig. 18) includes a collision detection module 86 that detects collisions between the user- 
controlled simulated surgical tools and the simulated components of the virtual body 
environment or between two or more simulated components of the body environment. The 
dynamic simulator also includes a contact force modeling module 88 that models the contact 
force between the simulated surgical tools and the components of the virtual body 
environment or between two or more simulated components of the virtual body environment, 
and provides an output to a movement modeling module 90 that models and computes the 
response or movement of the virtual body environment to the forces resulting from this 
contact. The contact forces are also applied to the surgical tools through the active force 
feedback devices. 

With the surgical simulation system of the invention, when the user of the surgical 
simulation system touches any component of the virtual body environment with the simulated 
surgical tool, the user will feel it at the surgical tool and the computed interaction forces will 
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prevent the user from doing something that is not possible such as, for example, pushing the 
simulated surgical tool through a simulated organ without resistance. One problem with prior 
art surgical simulators that do not include any of active force-feedback devices, collision 
modeling, contact force modeling and movement modeling is that a simulated object 
occupying a same physical space of another simulated object will pass completely through 
one another as though they are not there. Therefore, an advantage of the surgical simulation 
system of the invention is that the surgical simulation system of the invention provides the 
user tangible interaction with the virtual body environment in the form of active force- 
feedback and visible virtual body environment movement in response to interaction forces. 

Referring to Fig. 9, the simulation system processor 12 of the system 10 of the 
invention also includes a surgical skill evaluation module 64 (See Fig. 10) that performs a 
surgical skill evaluation process 61 for measuring, recording, evaluating and reporting the 
surgical performance of the user of the simulated surgical procedure. In particular, the 
measured surgical tool positions, orientations, and forces that occur during the simulated 
surgical procedure are monitored by this module and recorded (Step 63) in a programmable 
storage media associated with the system 10 of the invention. The stored information is used 
by the surgical skill evaluation module for detennining and providing a raw score of the user's 
performance (Step 65) based on physical data such as, for example, time information, force 
information, and position, orientation information of the simulated surgical tools, the virtual 
body environment and the mechanical body part, which is provided to the surgical skill 
evaluation module on lines 122, 124, 126, 128 and 130. (See Fig. 10.) The stored 
information is also compared to stored performance data of a population of practitioners in the 
field (Step 67) with experience ranging from inexperienced to "expert", who have also 
performed the simulated surgical procedure. The user's performance is also compared to a 
theoretical ideal performance (Step 69). All of this comparison data is used to determine a 
percentage ranking of the user's performance (Step 71). 

The raw score and percentage ranking provide detailed information to the user that is 
useful for evaluation of the user's knowledge and skill and which, in turn, can be used for 
education, training, and/or accreditation of the user of the system. In particular, as will be 
discussed in greater detail below, the force, position, orientation, and velocity of the surgical 
tools, the position and orientation of the elements of the mechanical body environment and the 
virtual body environment, can be used to evaluate the performance of the user. It is to be 
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appreciated that the output of the information to the user can be provided in a plurality of 
different ways such as, for example, on the visual display 14 (see Fig. 4) in a graphical format 
With the surgical simulation system of the invention, the user can compare his or her 
performance to that of practitioners in the field including an "expert", and the information can 
be portrayed to the user in a plurality of different manners. Thus, the surgical simulation 
system of the invention can be used for education, training and accreditation of the user on a 
surgical procedure. 

Fig. 10 illustrates an embodiment of the surgical skill evaluation module 64 of the 
surgical simulation system of the invention. The surgical skill evaluation module includes an 
arthroscopic camera navigation module 1 14, a surface damage measurement module 1 16, a 
damaged tissue identification module 1 18, a tool triangulation accuracy module 120, an 
accrued time measurement module 1 1 7 and a peak force measurement module 1 19. The 
surgical skill measurement module and in particular, modules 1 14-120 within the surgical 
skill measurement module, use the following information: the position and orientation of the 
simulated surgical tools on line 122; the position and orientation of the virtual body 
environment on line 124; the position and orientation of the mechanical body part on line 126; 
an indication of time on line 128; and the forces of interaction between the simulated surgical 
tools and the virtual body environment on line 130. This information is used by the 
arthroscopic camera navigation module to measure the ability of the user to find certain 
regions of anatomy in the knee with the camera, by the surface damage measurement module 
to measure the surface damage to the virtual body environment with the cameras, probes, or 
other tools, by the damaged tissue identification module to measure the ability of the user to 
locate and identify damaged tissue by touch, by the tool triangulation accuracy module to 
measure the accuracy with which the user can place a probe or other tool on a specific point 
inside the knee using the arthroscopic camera point of view, by the accrued time measurement 
module to determine the time the user takes to perform the simulated task, and by the peak 
force measurement module to measure the peak force applied by the user to the virtual body 
environment. 

Fig. 1 1 illustrates a method performed by the arthroscopic camera navigation module 
1 14 (See Fig. 10) for determining an ability of the user of the surgical simulation system to 
move the arthroscopic camera to certain regions within the mechanical knee. In particular, in 
Step 1 32, the user moves the arthroscopic camera to what is believed to be the desired 
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position. In Step 1 34, the module determines the position and orientation of the camera and 
provides the position information on line 135. The camera navigation module compares the 
position and orientation of the camera, provided on line 135, with the position of the virtual 
anatomy provided on line 124 (Step 148) to determine a relative position of the camera to the 
virtual anatomy, which is provided on line 149. Certain preferred camera positions for 
viewing certain anatomical structures have been pre-computed, stored in memory, and are 
supplied on line 151. These preferred positions are compared to the relative position of the 
camera on line 149, to determine a distance error between the relative position of the camera 
and the preferred position (Step 152), which is provided as an output on line 1 54. A 
component of the user's overall score is based upon this comparison result 

Another component of the user's overall score is based on the distance the tip of the 
camera travels in moving to the relative viewing position. In particular, the arthroscopic 
camera navigation module measures the distance the camera is moved by the user to the 
relative position (Step 156). In general, if the tool is moved directly to the relative position, 
then the distance traveled by the tip of the camera is mimmized and this component of the 
user's overall score is maximized. In contrast, if the user is less proficient with his or her 
movement of the tool by wandering around with the tool, the distance traveled by the camera 
is increased and this component of the user's overall score is lower. The distance traveled is 
provided on line 161. 

Another component of the user's overall score is the total accrued time it takes the user 
to perform the simulated task. The arthroscopic camera navigation module measures the total 
time the user takes to move the camera to the relative position (Step 158). Accordingly, if the 
tool is moved directly to the desired position, then the total time is minimized thereby 
maximizing this component of the user's overall score. In contrast, if the camera is not 
moved directly to the relative position, then the overall time is increased and this component 
of the user's overall score is lowered. The total time is provided by the arthroscopic camera 
navigation module on tine 163. The distance error on line 154, the total distance traveled on 
line 161, and the total time on line 163 are then multiplied by corresponding scaling factors 
(Steps 160, 162, and 164) and summed (Step 165) to provide an overall score of the user on 
line 167. 

The tool triangulation accuracy module 120 (see Fig. 10) uses similar steps as the 
camera navigation module to determine the ability of the user to visually guide the tool to a 
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given target within the virtual anatomy, using the arthroscopic camera for navigation. In 
particular, the user's overall score is based upon a relative tool position to certain pre- 
computed tool positions, a distance traveled by Ihe tool, and the total time it takes the user to 
perform the simulated task. Therefore, the tool triangulation accuracy module includes all the 
same steps that are used by the method of the arthroscopic camera navigation module, 
wherein the user is asked to touch certain target objects in the knee with the surgical tool, 
while maintaining the camera viewpoint with the simulated arthroscopic camera. As 
discussed with respect to Fig. 1 1, the position and orientation of the tool is measured and 
compared to the preferred tool position to determine an overall target distance error (Steps 
134-152). In addition, the total distance moved by the tool is measured (Step 156), and the 
total time to perform the task are measured (Step 158) and provided on lines 161 and 163. As 
discussed above with respect to Fig. 1 1, each of the distance error, the total distance, and the 
total time are multiplied by scaling factors (Steps 160, 162, and 164) and summed (Step 165) 
to yield an overall user's score on line 167. 

The damaged tissue identification module 1 18 (see Fig. 10) also includes some of the 
steps as illustrated in Fig. 11. In particular, the system of the invention simulates damaged 
tissue that looks normal but may have different tactile properties such as, for example, being 
softer or less compliant than that of healthy tissue. The user of the system can therefore only 
find the damaged tissue by feeling it Therefore, the user of the system moves the surgical 
tool until the damaged tissue is found (Step 132) and then by, for example, pushing a button 
or squeezing a trigger on the surgical tool, the damaged tissue identification module measures 
the relative position of the tool with respect to the virtual body environment, compares the 
relative position of the tool to the preferred damaged tissue position (Steps 134-152), and 
outputs an error distance on line 154. A component of the user's overall score is based on this 
overall error distance. As illustrated above with respect to Fig. 11, other components of the 
user's overall score may be the total distance traveled by the user to locate the damaged areas 
and the total time accrued to perform the task. Still another component of the user's overall 
score that was not discussed above with respect to Fig. 1 1 may be whether the user found all 
of the damaged area. In particular, the damaged tissue areas located by the user can be 
compared with all of the preferred damaged tissue areas to determine whether all have been 
found, and this can also be a component of the user's total overall score. 
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stored in the main memory and that controls the overall operation of the surgical simulation 
system and each of the modules to be described below. The processor further includes a 
display controller 72 that controls display of the simulated surgical tools and the virtual body 
environment on the 3-D graphics display 14 (see Fig. 4). Moreover, the processor may 

5 include: a 3-D graphics simulator 74 that renders, in real-time, a texture mapped virtual body 
environment and the simulated surgical tools or the 3-D graphics simulator may be part of a 
separate 3-D graphics workstation 13 (see Fig. 4). The processor further includes a force 
feedback interacter 76 that applies forces to the surgical tools 20, 22 via the force feedback 
devices 16, 18 (see Figure 3); and the dynamic simulator 78 that simulates interaction of the 

1 0 virtual body environment and the simulated surgical tools, as will be described in greater 
detail below. 

Referring now to Fig. 14, an embodiment of the 3-D graphics simulator 74 is 
illustrated. The 3-D graphics simulator includes a 3-D object modeler 80 that includes 3-D 
object models of any of bones, joints, body tissues, organs, cartilage, muscle tissues of the 

15 virtual body environment, as well as 3-D object models of the surgical tools. The 

3-D object models include geometry used for visual display that may be comprised of 
polygonal models, NURB models, or other computer graphic geometry modeling techniques 
known to those skilled in the art. The 3-D object graphical models include parameters for size, 
texture, geometry and topology of the simulated objects of the body environment. The 3-D 

20 graphics simulator also includes a texture mapping circuit 82 that applies texture mapping to 
the 3-D object models of the virtual body environment so that the virtual body environment 
and the components of the virtual body environment have a realistic appearance. Such texture 
mapping circuits are known to those of skill in the art. The 3-D graphics simulator further 
includes a lighting and point of view modeler 84 supporting lighting effects such as specular 

25 lighting, ambient lighting, and reflected lighting and supporting arbitrary views of the 3-D 
simulated environment through arbitrary positioning of the viewport. These lighting and 
point of view models are known to those of skill in the art. In the preferred embodiment, 
these lighting and point of view models are used to simulate a realistic surgical simulation of 
the virtual body environment, the components of the virtual body environment and the 

30 simulated surgical tools from the point of view of a simulated camera/arthroscope inserted 
into the mechanical leg. The point of view of the arthroscope is calculated by the 3-D 
graphics simulator's receipt of the position, orientation information of the body environment 
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on line 122, the position, location information of the surgical tool provided on line 124, and 
the position, orientation of the mechanical body part information provided on line 126. This 
information is used by the lighting, point of view module to determine perspective of the 
arthroscope with respect to the virtual body environment The relative position of the 
arthroscope to the virtual body environment is also used to determine simulated contact forces 
between the simulated arthroscope and the virtual body environment as discussed below. 

As was discussed above, the mechanical body part 47 (see Fig. 7) of the invention 
includes a resilient sheet underneath 214 a more compliant skin-like surface 212. Miinmally 
invasive tools when inserted into a portal 210, use the portal as a pivot so that when a handle 
of the tool 20 is moved in one direction on the outside of the body, the tool pivots about the 
portal so that the tool tip on the inside of the body moves in the opposite direction. This 
reversal in cause and effect is one reason why minimally invasive surgery is so challenging to 
learn. The skin material on the mechanical leg 47 of the invention acts as such a supportive 
pivot for the surgical tools 20, which is important for emulating the forces of interaction 
between simulated objects inside the leg and the tools. The pivot is an important part of this 
invention for replicating forces that would be present if the tool was actually touching an 
object inside the leg. This pivot is also an important part of facilitating attachment of the 
surgical tool to the force feedback devices 16, 18 (see Fig. 4), at the part of the surgical tool 
21 1 (see Fig. 6) that remains outside the mechanical body part, thus enabling arbitrary 
placement of the tool and manipulation of the surgical tool and/or mechanized leg or body 
part. 

Referring to Fig. 15, it is illustrated that the surgical tool 20, when placed in the 
interior of the mechanical body environment through an arbitrary portal 210 in the synthetic 
skin 212, can interact with simulated organs and tissues in such a way that multiple simulated 
interaction forces F ol , and F o3 from the simulated organs and tissues, can be applied to the 
surgical tool. These simulated forces can be applied at any place on the tool including in 
particular, at the tip or along the shaft of the tool. A net result of these multiple interaction 
forces can be equivalently represented by interaction force F„ and a moment about a well- 
defined point on the tool, such as at the portal or pivot point of intersection with the synthetic 
skin. Accordingly, a force feedback device that is used to mimic these multiple forces of 
interaction typically must produce some or all of these resulting forces and moments. 
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Referring now to Fig. 16, forces applied to the tool from the force feedback device 
must be equivalent to the interaction forces from the internal organs. It has been discovered 
with the present invention that these interaction forces to be simulated, can be decomposed 
into one component that acts along an axis of the tool F„ and another component that acts 
transverse to the axis of the tool F^. As illustrated in Fig. 16, these interaction forces include 
forces from the skin F s , including axial components F ra and transverse components F*. Also 
shown in Fig. 16 is the applied force from the force feedback device, F^, and its components, 
Faa, and F ff(tt , that is designed to emulate the interaction forces between the tool and the 
internal tissues. The force feedback forces are applied to the tool at the point 2 1 1 (see Fig. 6), 
that is some distance, L, from the pivot point. 

It is a novel discovery of this invention, that the interaction forces, M OT and F^ can be 
simulated with a 3-D force vector. In particular, the transverse components of the applied 3-D 
force vector are used to emulate the moment, and the axial components of the applied 
3-D force vector are used to emulate the axial component, F^ according to the following 
equations: 

F<n» = F ffda 

H„ = F ffd ,*L 

The role of the synthetic skin in these equations becomes apparent when we consider 
that the skin must support all transverse components of force according to the following force 
balance equation: 

F^ + F. + F. + Ffc-O 

where F ta is the transverse component of force applied by the user's hand, which also 
is applied to the tool. Therefore, the skin must be sturdy enough to support these transverse 
forces without significant deformation or distortion which would decrease the fidelity of the 
simulated forces. 

In particular, it is a novel aspect of this invention to simulate the forces of interaction 
between the surgical tool and the simulated objects inside the mechanical body environment 
with a force feedback device that need only apply a 3-D force vector to the physical tool. It is 
an advantage of this invention that the force feedback device need not apply a moment or 
torque to the physical tool. This is important according to the present invention, because force 
feedback devices that apply a moment to the tool at the center of rotation of the tool (the 
portal) may require a motor to be located at that point. This either necessitates that the motor 
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be fixed to the portal or that the motor be attached at the end of a linkage of the force feedback 
device, which significantly increases the mass, inertia, and complexity of the force feedback 
device and which will lower its performance. This makes the simulation of these interaction 
forces according to the present invention less complex. 

Another advantage of the system of the present invention is that the user can arbitrarily 
choose and change the portal 210 (see Figs. 6-7) for insertion of the niinimally invasive 
surgical tool. In particular, with the realization of the present invention that the multiple 
simulation forces can be simulated with a 3-D force vector applied to the tool at the point 21 1 
that remains outside of the mechanical body part, and with the synthetic skin layer of the 
mechanical body part, the user can arbitrarily choose and change the portal for insertion of the 
surgical tool. This allows the user to easily experiment with the effect of different portal sites. 
This is especially important to the user, when the user is creating a portal site with an incision 
as opposed to using a natural orifice of the mechanical body part For this situation, the 
choice of the correct portal location for incision is one of the skills that the user must learn. 
Accordingly, it is an advantage of the present invention that the appropriate choice of the 
portal location can be experimented with and learned according to the surgical simulation 
system of the invention. Therefore, it is an advantage of the surgical simulation system of the 
present invention that an orifice or incision into the mechanical body part does not have to be 
instrumented in any way. In particular, the user can arbitrarily choose it. 

The 3-D graphics simulator 74 (see Fig. 14) also includes a deformation module 85 
that efficiently modifies the geometry of elements of the virtual body environment in real-time 
to represent deformation that occurs during interaction with the simulated surgical tools or 
other simulated objects. The deformation module modifies the geometric models in the 
region near the application of an interaction force with the surgical tool or other element of the 
body environment 

In one preferred embodiment of the surface deformation module 85, this interaction 
force is applied at one or more points on the simulated 3-D object such as, for example, the 
point at which the meniscus is poked with a simulated probe. In other words, for the 
arthroscopic surgical procedure, the surface deformation module modifies the point on the 
meniscus that is touched by the surgical probe to follow the probe as the probe is pushed into 
the meniscus. The geometry model in the vicinity of the touch point that represents the 
simulated body meniscus such as, for example, the vertices of a polygonal meniscus model, 
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aremodifiedaccordingtoadefonnaaonftoctton. As Olushated in Fig. 17A, uk deformation 
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representations of two simulated objects. These collision detection techniques are generally 
known to those of skill in the art. If the two simulated objects are found to be 
interpenetrating, then the collision detection module determines the surface features of the 
simulated objects that are colliding and the amount of interpenetration is calculated. 

When two simulated 3-D objects are found to be in contact, then the contact force 
modeling module 88 will compute interaction forces between the simulated objects in order to 
simulate interaction of the two simulated objects. In particular, equal and opposite contact 
forces are applied to each simulated object in accordance with Newton's Third Law. The 
contact force modeling module determines the interaction forces based upon the relative 
position and velocity of points on the surface of each simulated object. The position and 
velocity information is provided by the surgical tools kinematics module 93 and the simulated 
organ kinematics module 9 1 . The contact forces are computed with force models that 
combine the relative position and velocity information with spring elements, dissipative 
elements, and frictional elements as is generally known to those of skill in the art The spring 
elements determined by the force models are a function of the relative position of the 
contacting surfaces of the simulated 3-D objects. The dissipative elements are a function of 
relative velocity of the contacting surfaces. According to this embodiment of the dynamic 
simulator of the invention, the net forces due to the spring and dissipative elements are 
constrained to satisfy friction models such as, for example, Coulomb friction, static friction, 
and kinetic friction. The friction models provide for slipping of one contacting surface of a 3- 
D simulated object relative to another contacting surface of the other 3-D object. The friction 
models constrain the contact forces such that they obey certain relationships between the 
simulated objects. These friction models are generally known to one of skill in the art The 
contact force modeling module then provides an output to the movement modeling module 90 
indicating the contact forces that result from the simulated contact between the simulated 
objects of the virtual body environment. 

The movement modeling module 90 determines movement of the simulated objects in 
response to forces provided by the contact force modeling module 88, that arise from contact 
between any of the simulated objects and/or forces due to gravity. As discussed above, the 
simulated objects have physical attributes such as, for example, mass, inertia, energy and 
momentum. Accordingly, the simulated objects move realistically in response to contact 
forces; this movement is determined by the movement modeling module according to 
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Newton's Second Law of Motion. The movement modeling module includes the physical 
description of the simulated objects such as each object's mass, inertia, size and its joints or 
degrees of freedom. The movement modeling module also includes an acceleration modeling 
module 92 that determines the acceleration of the simulated objects and degrees of freedom, 
such as, for example, translation and rotation of the simulated objects in response to the 
application of the above-described external forces and/or a lack of such external forces. In 
particular, referring to Fig. 19, there is illustrated a method of determining the movement of 
the simulated objects according to the invention. The movement modeling module receives 
the interaction force information (Step 184) from the contact force modeling module. The 
movement modeling module computes the acceleration (Step 186) of the simulated objects. 
The acceleration of objects is integrated with time to determine a velocity (Step 188) and a 
position (Step 190) of the simulated objects through a numerical integration process generally 

known to one of skill in the art 

According to the invention, a surgical simulation system and method is provided for 
learning, practicing, experiencing and evaluating nrinimally invasive surgical procedures such 
as arthroscopy and laparoscopy. With the method and system of the invention, the need for 
physical models or live patients is eliminated and a life-like simulation is provided such that 
the user sees and hears a life-like simulated surgical environment and dynamic situations in 
real-time using a mechanical body part, 3-D computer generated graphics and sound, and such 
that the user feels the interaction with the virtual body environment and simulated surgical 
tools. 

A big advantage of the invention is that these minimally invasive procedures can be 
practiced including decisions that are necessarily learned such as, placement of the incision or 
portal of the minimally invasive tool. 

Another advantage of the surgical simulation system and method of the invention is 
that it will help to improve medical education by providing a plurality of life-like surgical 
situations to the user to experience, practice and perfect The system and method may be used 
to control and standardize training regimen. With the system and method of the invention, 
students can learn by practicing. In particular, students can be presented with lessons that are 
commensurate with their skill level and students can learn to adapt procedures for a range of 
anatomical variations and surgical conditions. Moreover, students will be able to repeat the 
simulated surgical procedures until they master them without fear of any harm to a patient 
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The surgical simulation system and method of the invention also makes it possible to 
objectively evaluate the surgical skills of a student. The detailed information provided by the 
system and method such as, for example, position, velocity and force information makes it 
possible to measure the tool accuracy, tissue damage, and surgical techniques of the student 
The students will also be able to review their performance and compare it to that of "experts" 
in the field who have performed the same procedure. Moreover, by using the recorded 
performance of the "expert" as a standard, the surgical simulation system and method of the , 
invention further enhances the teaching capacity of experts beyond the one-to-one 
apprenticeship that is the standard today. 

Another advantage of the surgical simulation system and method of the invention is 
that it can also be used with traditional training techniques to improve the quality and reduce 
the costs of surgical education by insuring that students are prepared to make the best possible 
use of valuable time that is provided to the students in the operating room. 

Still other advantages of the surgical simulation system and apparatus of the invention 
include: surgical training can be provided without the use of human patients, hospital space or 
animals; the surgical simulation system and method will help develop and train the physical 
motor skills of the student, the perceptual tasks of the student and the cognitive decision- 
making of the student, and the surgical training system and method will allow medical 
students to practice routine procedures, encounter patients with rare medical conditions or 
unexpected complications and to practice techniques to handle each of these situations. 
Accordingly, in contrast to accepted practice where only experienced surgeons are exposed to 
such a wide range of conditions and complications that can occur, the surgical simulation 
system and method of the invention will provide a wide range of conditions and complications 
to any user. In addition to the training provided by the surgical simulation system and method 
of the invention, the system and method of the invention can also be used to assess the 
surgical performance of the student and play a roll in certification of the student for surgical 
procedures. 

Having thus described at least the preferred embodiment of the invention, various 
alterations, modifications, and improvements will readily occur to those skilled in the art. 
Such alterations, modifications, and improvements are intended to be part of this disclosure, 
and are intended to be within the spirit and scope of the invention. Accordingly, the foregoing 
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description is by way of example only and the invention is limited only as defined in the 
following claims and the equivalents thereto. 
What is claimed is: 
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1 An interactive, surgical simulation system comprising: 

a physical body environment that replicates a portion of a human body and includes a 
synthetic skin layer, the physical body environment allowing insertion of a surgical tool 
through the synthetic skin layer into the physical body environment at an arbitrary location 
selected by a user, 

a body environment simulator that simulates a virtual body environment within the 

physical body environment; 

a surgical tool simulator that simulates a simulated surgical tool within the virtual 
body environment at a location inside the virtual body environment corresponding to the 
arbitrary location, and is simulated to move corresponding to the surgical tool that is 
manipulated by the user of the surgical simulation system; 

an interaction simulator that simulates interaction between the virtual body 
environment and the simulated surgical tool including any force or torque exchanged between 
the simulated surgical tool and the virtual body environment, and for providing the simulated 
interaction as a 3 -dimensional force vector; 

a force feedback device attached to the surgical tool at a point outside of the physical 
body environment, that applies the 3-dimensional force vector corresponding to the simulated 
interaction, to the surgical tool at the point outside of the physical body environment; and 

avisual display device for displaying the virtual body environment, the simulated 
surgical tool and the simulated interaction. 

2 The surgical simulation system as claimed in claim l,whereinthe synthetic skin layer 
includes a resilient sheet material disposed beneath the synthetic skin layer, the resilient sheet 
material providing a stable pivot that facilitates the surgical tool rotating about the stable pivot 
when the three-dimensional force vector is applied to the surgical tool, and thereby 
eliminating the need for a motor or torque producing device to be located at the stable pivot. 

3 The surgical Simulation system claimed in claim 1, further comprising a perspective 

determining device that determines a position of me simmated^ 

virtual body environment, and that provides to the visual display device, information for 
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the physical tool within the physical body environment 

Wy environment, or movement of the virtual body envimnment 
environment is jointed to move like a jointed body pari 

S The surgical simulation system as Caimed in e!aim 5, wherein ft. pbysica. body 
within the physical body environment. 

^moment includes sensors to measure physical forces apptied ro tire phy»cnl bod, 
environment. 

m vironment with movement of the physical body environment; and 
environment. 

„ hlc^Llnclndhrg 3-0 B eometi,c object models and com^simma.stire 
in 3-D. 
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10. The surgical simulation system as claimed in claim 9, wherein the 3-D graphics 
simulator includes a texture mapping circuit that provides texture to the virtual body 
environment so as to represent a realistic body environment and components of the virtual 
body environment 

11. The surgical simulation system as claimed in claim 10, wherein the virtual body 
environment includes biological tissue simulated in various states of decomposition or 
disrepair. 

12. The surgical simulation system as claimed in claim 1, wherein the body environment 
simulator and the surgical tool simulator include lighting models and point of view models for 
the perspective of the physical tool within the physical body environment 

13. The surgical simulation system as claimed in claim 1, wherein the interaction 
simulator includes a collision detector that determines whether any collision occurs between 
the simulated surgical tool and the virtual body environment 

14. The surgical simulation system as claimed in claim 13, wherein the interaction 
simulator further includes a contact force modeling module that determines an exchange of 
force between the simulated surgical tool and the virtual body environment. 

15. The surgical simulation system as claimed in claim 1 , wherein the interaction 
simulator includes a movement modeling module and a surface deformation module that 
determine movement and deformation of any of the simulated surgical tool and the virtual 
body environment in response to any of the movement of the physical tool, the movement of 
the physical body environment and the movement of the virtual body environment. 

16. The surgical simulation system as claimed in claim 15, wherein the movement 
modeling module distorts a surface of a component of the virtual body environment to provide 
a deformed virtual component according to a relative position of the simulated surgical tool to 
the component of the virtual body environment. 
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17. The surgical simulation system as claimed in claim 16, wherein the movement 
modeling module further includes an acceleration module that determines an acceleration of 
the virtual body environment in response to contact between the simulated surgical tool and 
the virtual body environment. 

18. The surgical simulation system as claimed in claim 1 , further comprising a surgical 
skiU module that records any of a surgical tool position and orientation, the virtual body 
environment position and orientation, interaction forces between the virtual body environment 
and the simulated surgical tool, an elapsed time of a simulated procedure by the user, and a 
total distance traveled by the surgical tool. 

19. The surgical simulation system as claimed in claim 18, wherein the interaction forces 
include forces at the physical tool and the point of application of the force between the virtual 
body environment and the simulated surgical tool, as a result of the user performing the 
simulated medical procedure on the physical body environment 

20. The surgical simulation system as claimed in claim 18, wherein the surgical skill 
measurement module includes performance data of others, including idealized performance 
data indicative of an ideal performance of the simulated medical procedure. 

21. The surgical simulation system as claimed in claim 20, wherein the surgical skill 
measurement module compares the performance data of others with the data of the simulated 
medical procedure by the user, to provide comparison data that is presented to the user. 

22. The surgical simulation system as claimed in claim 21 , wherein the surgical skill 
measurement module analyzes the comparison data to provide an assessment of the simulated 
surgical procedure by the user. 
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23. A method of simulating minimally invasive surgery, comprising the steps of: 

providing a physical body environment that replicates a portion of a human body and 
includes a synthetic skin layer, the physical body environment allowing insertion of a surgical 
tool through the synthetic skin layer into the physical body environment at an arbitrary 

location selected by a user; 

simulating a virtual body environment within the physical body environment; 

simulating a simulated surgical tool within the virtual body environment, at a location 
inside the virtual body environment corresponding to the arbitrary location, and that moves 
corresponding to the surgical tool that is manipulated by the user of the surgical simulation 
system; 

simulating interaction between the virtual body environment and the simulated 
surgical tool including any force or torque exchanged between the simulated surgical tool and 
the virtual body environment, and providing the simulated interaction as a 3-dimensional 
force vector, 

applying the 3-dimensional force vector corresponding to the simulated interaction, to 
the surgical tool at a point outside of the physical body environment; and 

displaying the virtual body environment, the simulated surgical tool and the simulated 

interaction. 

24. The method as claimed in claim 23, wherein the step of providing the physical body 
environment includes providing a resilient sheet material disposed beneath the synthetic skin 
layer, the resilient sheet material providing a stable pivot that facilitates the surgical tool 
rotating about the stable pivot when the three-dimensional force vector is applied to the 
surgical tool, and thereby eliminating the need for a motor or torque producing device to be 
located at the stable pivot. 

25. The method as claimed in claim 23, further comprising the step of determining a 
position of the simulated surgical tool with respect to the virtual body environment, and 
displaying the virtual body environment and the simulated surgical tool from a perspective of 
the surgical tool within the physical body environment. 
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26. The method as claimed in claim 23, including simulating any of movement, collision 
and force between the virtual body environment and the simulated surgical tool, in response to 
any of movement of the physical tool, movement of the physical body environment, and 
movement of the virtual body environment. 

27. The method as claimed in claim 23, including measuring physical forces applied to the 
physical body environment and simulating the virtual body environment responsive to the 
measured physical forces. 

28. The method as claimed in claim 23, including measuring a position of the jointed body 
environment with movement of the physical body environment and responsive to the 
measured position of the jointed body environment, simulating the virtual body environment 

29. The method as claimed in claim 23, further comprising simulating the virtual body 
environment and the simulated surgical tool within the virtual body environment in 3-D. 

30. The method as claimed in claim 29, further comprising texture mapping the virtual 
body environment so as to represent a realistic body environment and components of the 
virtual body environment. 

31. The method as claimed in claim 30, further comprising simulating biological tissue in 
various states of decomposition or disrepair. 

32. The method as claimed in claim 23, further comprising applying the 3-dimensional 
force vector to the physical tool at a point outside the physical body environment, to simulate 
the forces of interaction between the physical tool and the virtual body environment within the 
physical body environment. 



33. The method as claimed in claim 23, further comprising determining whether any 
collision occurs between the simulated surgical tool and the virtual body environment. 
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34. The method as claimed in claim 33, further comprising detennining an exchange of 
force between the simulated surgical tool and the virtual body environment. 

35. The method as claimed in claim 23, further comprising determining movement and 
deformation of any of the simulated surgical tool and the virtual body environment, in 
response to any movement of the physical tool, the movement of the physical body 
environment, and the movement of the virtual body environment. 

36. The method as claimed in claim 35, further comprising distorting a surface of a 
component of the virtual body environment to provide a deformed virtual component, 
according to a relative position of the simulated surgical tool to the component of the virtual 
body environment 

37. The method as claimed in claim 36, further comprising detennining an acceleration of 
the virtual body environment in response to contact between the simulated surgical tool and 
the virtual body environment 

38. The method as claimed in claim 23, further comprising recording any of a surgical tool 
position and orientation, the virtual body environment position and orientation, interaction 
forces between the virtual body environment and the simulated surgical tool, an elapsed time 
of a simulated procedure by the user, and a total distance traveled by the surgical tool. 

39. The method as claimed in claim 38, further comprising recording the interaction forces 
at the physical tool and the point of application of the force between the virtual body 
environment and the simulated surgical tool, as a result of the user performing the simulated 
medical procedure on the physical body environment 

40. The method as claimed in claim 39, further comprising comparing the performance of 
others with the performance of the simulated medical procedure by the user. 

41. The method as claimed in claim 40, further comprising analyzing the comparison data 
to provide an assessment of the simulated surgical procedure by the user. 
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